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(57) An image sensor device comprising a silicon 
substrate having a plurality of CMOS circuit formed ther- 
eon, including a pixel array having a plurality of rows 
and a plurality of columns, a row addressing circuit op- 
eratively connected to each the pixel array and the tim- 
ing control circuit, the row addressing circuit having a 
row bus that provides address lines to each row in the 
pixel array, a column addressing circuit, a pixel timing 
circuit, a timing control logic block, a signal processing 
circuit, and an interface circuit coupled to external com- 
putational means for provision of address and control 
signals to the sensor device, the interface circuit being 
operatlvely coupled the timing control logic, the pixel 
timing circuit, the row addressing circuit and the column 
addressing circuit. 



it! W 
Oz to 



[ 1 



Ml 



5G 



f ---i-i-{-{--t--l — -/r- 



SERfAL INTERFACE & CONTROL 



PROGRAM 
STORAGE 



\ COLUMN ADDRESSfNG~] 



Cfc Q. 



60 t 



3J 





P 12 - 
/ 


1 

APS 


g 

r 
1 

PIXEL ARRAY 



TIMING 
CONTROL 



£ :^ 
ct ^ <o 
S o o 

5^3 :s lo t ^ 

1 M i I 



H 



SIGNAL PROCESSING- 
CDS. FPN REMOVAL, 
PGA. ADC 



13 



is? 

Sis 
I t I 



5 
ct 



1 K DIGITAL 

J DATA 

OUT 



FIC. 1 



Printed by Jouve, 75001 PARIS (FR) 



3NSDOCID: <EP ^0932302A2_L> 



1 



EP 0 932 302 A2 



2 



Description 

FIELD OF THE INVENTION 

[0001] The invention relates to CMOS based innage 
sensors and more particularly to camera on a chip de- 
signs that are based on CMOS active pixel sensors. 

BACKGROUND OF THE INVENTION 

[0002] The are numerous prior-art solid state imaging 
devices. Most common are Charge Coupled Device 
(CCD) based sensors. CCD sensors within the prior art 
lack very large scale integration (VLSI) that is employed 
in modern CMOS devices and in making the cameras 
that employ these CCD sensors require separate inte- 
grated circuits to provide timing, control and clock dnver 
circuitry, therefore, resulting in larger and more expen- 
sive to produce cameras. 

[0003] Additionally within the prior art are a number 
image sensing devices that have employed CMOS tech- 
nology to make the solid state imagers. Among these 
prior art devices are teachings of integrating the analog 
to digital converter on the same chip as the image sen- 
sor. A CMOS image sensor of this type is described in 
a paper entitled "An Addressable 256 x 256 Photodiode 
Sensor Array With An 8-Bit Digital Output" in Analog In- 
tegrated Circuits and Signal Processing 4, pp. 37-49 
(1993) by Janssen et al Other prior art devices have 
integrated on chip timing generation such as those de- 
scribed by Nixon et al in a paper entitled "256 x 256 
CMOS Active Pixel Sensor Cameraon-a-Chip", in 1996 
IEEE International Solid-State Circuits Conference Di- 
gest of Papers, p. 178-179. However, larger scale inte- 
gration is lacking within these prior art CMOS devices. 
[0004] The prior art is also silent towards a coordinat- 
ed design that allows for external digital interface that 
can control the image sensing device. 
[0005] It should be apparent from the foregoing dis- 
cussion that there remains a need within the art for a 
CMOS based imager that provides large scale integra- 
tion sufficient to provide timing, control and clock circuit- 
ry and a digital interface that can control the solid state 
imaging device. 

SUMtMARY OF THE INVENTION 

[0006] This invention addresses the aforementioned 
shortcomings within the prior art by providing an archi- 
tecture of a digital camera-on-chip, built using an active 
pixel sensor (APS) employing Complementary Metal 
Oxide Semiconductor (CMOS) technology. The ele- 
ments of the architectural design comprises Functional 
blocks and their respective interconnections required to 
operate the APS, which in the preferred embodiment 
employs a Pinned Photo Diode as its photodetector el- 
ement. Employing CMOS technology allows integration 
of timing, control, analog to digital conversion., and in- 



corporation of a digital interface onto a single chip with 
the photodetecting element array. 
[0007] The large scale Integration taught by the 
present invention allows features such as electronic 
s zoom and windowing capability. The features provided 
can be controlled by logic within a digital interface. A 
free running mode (default operational mode) provides 
a full resolution image that Is independent of the win- 
dowing/zoom logic therefore, not requiring control sig- 
10 nals from the digital interface. A Photometeric mode al- 
lows for selection of groups of pixels to determine the 
proper parameters such as exposure control. This ex- 
posure is effected by proper timing of the electronic 
shutter. A variety of sleep modes can provide power sav- 
^5 ings of varying degrees. 

[0008] The architecture of this CMOS APS comprises 
of the following connected blocks x-y addressable pixel 
array timing generation and external interface control 
logic, programmable Analog to Digital Converter (ADC) 
and associated signal processing circuitry. 
[0009] These and other features are provided by a 
CMOS based image sensor having 
[0010] An image sensor device comprising: 



2S a silicon substrate having a plurality of CMOS circuit 
formed thereon; 

a pixel array having a plurality of rows and a plurality 

of columns formed within the substrate; 

a timing control logic block formed within the sub- 

30 strate; 

a row addressing circuit formed within the substrate 
and operatively connected to each pixel array and 
the timing control circuit, the row addressing circuit 
having a row bus that provides address lines to 

35 each row in the pixel array; 

a column addressing circuit formed within the sub- 
strate; 

a pixel timing circuit fomned within the substrate; 
a signal processing circuit contained within the sub- 
40 strate; and 

an interface circuit coupled to external computation- 
al means for provision of commands directing the 
sensor to generate address and control signals to 
the sensor device, the interface circuit being oper- 
as atively coupled to the timing control logic, the pixel 
liming circuit, the row addressing circuit and the col- 
umn addressing circuit. 

BRIEF DESCRIPTION OF THE DRAWINGS 

so 

[0011] 

Fig. 1 is a functional block diagram of the integrated 
CMOS image sensor as envisioned by the present 
55 invention; 

Fig. 2 is a functional block diagram of a pnor art 

CCD camera; 

Fig. 3 is a block diagram of a camera employing the 
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camera on a chip sensor as envisioned by the 
present invention having sinrillar functionality to the 
diagram shown in Fig. 2 

Fig. 4 is a block diagram of the present invention 
addressing and read out of pixels in the pixel array; 
Fig. 5 is a diagram illustrating the addressing, mul- 
tiplexing and delivering of pixels to an analog digital 
converter 

[0012] For a better understanding of the present in- 
vention, together with other and further objects, advan- 
tages and capabilities thereof, reference is nr^ade to the 
following detailed description and appended claims in 
connection with the preceding drawings and description 
of some aspects of the invention. 

DETAILED DESCRIPTION OF THE INVENTION 

[0013] It has been discovered that a digital camera- 
on-chip can be built using an active pixel sensor (APS) 
employing Complementary Metal Oxide Semiconductor 
(CMOS) technology that integrates functions for timing, 
control, analog to digital conversion, and incorporation 
of a digital interface onto a single chip. 
[0014] Referring to Fig. 1, the elements of the archi- 
tectural design generally referred to as sensor 10, com- 
prise functional blocks and their respective Interconnec- 
tions required to operate an the APS pixel array 12. Ad- 
ditionally, the architecture of sensor 10 provides timing, 
control, clock and analog to digital conversion on a sin- 
gle CMOS chip. The present invention as seen in Fig. 1 
represents a block diagram that can be implemented by 
any of numerous design disciplines, such as hardware 
description language, or schematic entry. Preferably, 
the Implementation would be in CMOS to provide rea- 
sonable power consumption. A silicon implementation 
can be created through a design specification that is 
then used to create either a microprogrammed based 
implementation or a state machine based implementa- 
tion. The preferred embodiment is placed in silicon em- 
ploying a state machine written in VHSIC Hardware De- 
scription Language (VHDL). VHDL is employed as a 
form of electronic design automation. 
[0015] The pixel array 12, as seen in Fig. 1. is x-y ad- 
dressable forming two-dimensional array 12 of pixels. 
Various resolutions of array 12 are possible, such as a 
standard VGA resolution of 640 by 480 pixels. Each pix- 
el within the preferred embodiment typically comprises 
a photodetector (which in the preferred embodiment is 
a pinned photodlode), a transfer gate, a reset gate, a 
row-select gate and a source-follower amplifier. 
[001 6] The array 1 2 will convert incident light into pho- 
toelectrons which will reside within each photodetector 
in the array 1 2 as a stored charge version of the incident 
image focused upon the array 1 2 through a camera lens 
(not shown). Each pixel is selected for outputting the 
stored charge through row addressing 14 and column 
addressing 16 circuits that are integrated on sensor 10. 



The image is read out by having row select 14 provide 
the address for the selected row on row bus 15 which 
sets a common row bus to active state. 
[0017] The serial interface and control 50 allows for 

5 extemal microcontroller, or other means of computer 
program, to provide settings for the camera-on-a-chip 
based on operator or other automated algorithm. 
Among these settings are operations of the camera chip 
in specific modes such as light-intensity (photometer) 

10 measurement for setting exposure conditions. There is 
also a default power-up mode of the imager chip which 
runs it at a predetermined manner before any program- 
ming is performed. For example, when power is first ap- 
plied the chip would run in a full-resolution mode with 

15 an exposure timing consistent with room light condition. 
Th is default mode Is designed appropriately for each ap- 
plication intended. 

[0018] The serial interface and control logic 50 pro- 
vides controls used in the generation of sequencing sig- 

^0 nals such as x-y addresses, reset, sample, transfer and 
other signals generated by the timing control logic 60. 
A set of commands from the serial interface and control 
logic 50 is loaded into the appropriate control registers 
within the timing and control logic 60 to enable different 

^5 timing modes for integration and readout of the pixel ar- 
ray. The read-out of sensor 1 0 can be programmed at 
full resolution (all the pixels), an x-y windowed set of the 
pixels, or by subsampting (every other row) etc. The tim- 
ing logic control 60 also provides control signals for the 

30 Operation of the reading out of the sensor 10. 

[0019] The present Invention envisions that an Analog 
Digital Converter (ADC) may be included in the CMOS 
sensor 1 0. Also envisioned is providing an ADC external 
to sensor 1 0. This is simply a design choice that is made 

3S based on tradeoffs of the benefits and detriments of 
each of the designs. In either case it Is important that 
the ADC be placed in conjunction with the pixel read- 
out to ensure the digital bit output are meaningful and 
artifacts are minimized. 

40 [0020] The timing logic control 60 provides: (1) ad- 
dresses to the row addressing and pixel timing bus 14 
when every row is going to be readout in a full resolution 
mode; (2) addresses to the row addressing and pixel 
timing bus 14 when a subset of rows are going to be 

45 readout such as in every other or every fourth row as 
used in a subsampling mode that would typically be em- 
ployed in videoconferencing; and (3) the windowing ca- 
pabilities needed for addressing the subregions of pix- 
els. The timing and control logic 60 also controls elec- 

50 tronic shuttering mode control including: (1) the reset- 
ting of pixels relative to the frame rate; (2) control of in- 
tegration mode for each frame; (3) photometer mode 
which controls light sensitivity level and adjustment of 
electronic shuttering, this also includes the sensing of 

55 fluorescent light beat (i.e. 60Hz) and responsive control 
of the electric shutter that satisfies the relation ESS = 
" — r- seconds, where ESS is the electronic shutter 

<i iine_r3ie 

speed and n is potentially any factor of the line rate, 
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which in the preferred embodiment is 6OH2. 
[0021] Sensor 10 generally operates as follows: at a 
pre-determined time, a transfer signal is applied to alt 
the pixels In the row by a bus line within the common 
row bus to move the collected photoelectrons from each 
of the photodiodes within that row to a sensing node for 
each of the pixels. The sensing node for each of the pix- 
els is connected to a source follower amplifier input, 
which Is typical within the art of Active Pixel Sensors. 
The row bus also enables the reset signal to be applied 
to the pixels which connects the photodiode and/or the 
sensing node to a pre-set bias reset voltage. 
[0022] A double correlated sampling scheme can also 
be employed to output the reset level of the sensing 
node by first resetting it and then read the source-fol- 
lower output before transferring the photoelectrons. An- 
other output sample is taken after the photoelectrons 
are transferred from the photodiode. The difference in 
the two sampled signals represent a true reading 0I the 
amount of photoelectrons being generated by the light 
incident on that pixel. 

[0023] The column signal processing and ADC 18 al- 
lows the temporary storage In capacitors of the reset 
and signal levels of each pixel. These signals are then 
used by an ADC circuit to convert the difference of the 
signal levels to a digital representation. Depending on 
the speed of the particular ADC circuit employed, the 
digital values can be delivered outside the camera-on- 
a-chip at real lime rates to update the Image display or 
for motion capture by a frame storage. Optionally, a local 
on chip temporary digital storage could be used to buffer 
the data. This ADC circuit could be a single high-speed 
unit which can convert the number of pixels (total 
number of pixels times the frame rate) needed by the 
external device in real time. Alternatively, a per-column, 
or a per-n-column, slower ADC could be employed to 
convert all the analog pixel values in the rows in parallel. 
[0024] Fig. 2 illustrates a typical prior art CCD based 
system 75 showing the various devices used to imple- 
ment a digital camera. Microcontroller 72 provides the 
basic control signals for system 75. Timing and logic 
control 74, typically an ASIC device, will generate timing 
signals required by clock driver 77 which translates 
these signals Into the appropriate levels necessary to 
control CCD 76. The Microcontroller 72 programs the 
proper sequence for the imaging mode being used. Cor- 
related Double Sampling (CDS) 78 will generate the true 
value of the signal related to light Impinging on CCD 76. 
The true value of the signal Is then converted into a dig- 
ital signal by analog to digital converter 79. 
[0025] Fig. 3 illustrates the a system as envisioned by 
the present invention which has numerous advantages 
over the system shown in Fig. 2. Most of the functional 
blocks as seen in Fig. 2 are incorporated Into a single 
silicon camera on-a-chip system 10 of Fig. 3 with the 
exception Microcontroller 81 and interface ASIC 82. It 
will be understood by those skilled In the art that Micro- 
controller 81 and ASIC 82 can be combined into a larger 



ASIC device 80 having embedded with a controller. 
ASIC 82 provides interface with serial interface and con- 
trol logic 50 of the camera-on-chip system 1 0. Within the 
preferred embodiment, ASIC 82 provides digital 

5 processing of digital data that is output from the camera 
on-a-chip system 10. ASIC 82 provides faster data 
processing of image data because of the hardware ori- 
ented processing that is provided. ASIC 82 and control- 
ler 81 can also be formed as part of the camera-on-a 

10 chip system 1 0. A buffer memory 85 may be included to 
throttle the data from the (Analog to Digital Converter) 
ADC 86. 

[0026] Referring now to Fig. 4, a simplified block dia- 
gram of an embodiment of the present Invention illus- 
trating a pixel array 1 02 is shown comprising a 4 row by 
six column array of pixels. The illustration in Fig. 4 is 
intentionally simplified, the preferred embodiment of the 
present invention Is a 640 by 480 Image sensing device. 
Referring to Fig. 4, addressing circuit 114 Includes ad- 
dressing circuits for both row and column selection. The 
following discussion will detail the decoding of row ad- 
dresses which is a counter scheme (in the case of Fig. 
4 only a 2 bit counter), the decoding of column address- 
es is accomplished In a similar manner but is increment- 
ed differently. The row addressing portion and provides 
operating as well as timing and control signals (such as 
transfer, reset enable and select) to the specific row be- 
ing addressed. This addressing circuit can be built using 
logic gates such as NAND gates or It may be construct- 
ed as a shift register to sequentially address each row. 
[0027] Control signals that are generated by the tim- 
ing and control circuit 60 row and column addressing 
114 as well as sequencing of the per column signal 
processing. The timing and control circuit 60 is pro- 
grammed by the Microcontroller 81 as shown in Fig. 3 
using commands that traverse the serial interface and 
control 50 (as seen in Fig. 3) via the communication port 
to interface with sensor chip 10. Upon being addressed, 
a row of pixels will strobe the photoelectrons stored 
therein into the correlated double sampling (CDS) 116 
circuits which reduce the noise in the signal. Fig. 4 illus- 
trates a per-column ADC circuit 118 that detects the sig- 
nal that Is readout of the pixel array via a Correlated 
Double Sampling (CDS) 116 circuit. The"signal is then 
converted Into a digital word by the per-column ADC 
118. ADC 118 can perform this conversion either on a 
single column or on a set of the columns. The preferred 
embodiment employs 8 bit analog to digital conversion 
on a per basis for speed considerations. The ADC can 
be constructed using a successive approximation 
scheme that is well known in the art of ADC design. The 
per column (or per set of columns) arrangement allows 
the conversion of the pixel signals In parallel so that the 
conversion speed of the ADC is reduced to the line scan 
time. 

[0028] The timing signals required by the CDS 116 

and the ADC 118 circuits are generated by the Timing/ 
Control unit 60. For example the per column CDS 116 
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performs a reset sample before the transfer gate acti- 
vated with an entire row of pixels from the pixel array, 
this is followed by a signal sampling after transfer from 
the diode to the sensing node of each pixel. 
[0029] Referring now to Fig. 5, which is an illustration 
of an embodiment of the invention similar to that shown 
in Fig. 4 and identical circuits therefore, have the same 
reference indicators. In Fig. 5 however, the signals from 
each pixel of the addressed rows are supplied to signal 
multiplexer 126 where they are multiplexed and deliv- 
ered to a single or multiple high speed ADC 1 28 for con- 
version to digital words. In the example shown in Fig. 5 
the number of actual analog to digital converters used 
to implement high speed ADC 128 is small (such as for 
example either a single ADC or possibly multiple 
ADC's), the conversion rate is now the normal pixel data 
rate divided by the number of ADCs. 
[0030] The large scale integration taught by the 
present invention allows features such as electronic 
zoom and windowing capability The features provided 
can be controlled by logic within a digital interface. A 
free running mode (default operational mode) provides 
a full resolution image that is independent of the win- 
dowing/zoom logic therefor, not requiring control signals 
from the digital interface. A Photometeric mode allows 
for selection of groups of pixels to determine the proper 
parameters such as exposure control. A variety of sleep- 
er modes can provide power savings of varying de- 
grees. 

[0031] The imager array can be made color-sensitive 
by adding a color filter array (CFA) mosaic patterned in 
a predetermined manner (such as with a Bayer checker 
board) to give color rendition of the scene being imaged. 
[0032] The architecture implements a complete cam- 
era-on-chip. All the electronic control elements and sig- 
naling circuits are being integrated onto a single piece 
of silicon. This simplifies camera systems design in a 
way that the camera designer can concentrate on fea- 
tures (such as the human interface) rather than details 
of the timing required to capture an image with the chip. 
[0033] The integrated chip also is lower power an Im- 
portant consideration for portable application. 
[0034] Other aspects of the invention Include: the de- 
vice further comprising means for selecting a predeter- 
mined set of pixels from the imaging array contained 
within the timing control block; the device further com- 
prising means for performing electronic shuttering con- 
tained within the liming control block; the device further 
comprising means for selecting a subset of pixels for ex- 
posure determination within the timing control block; the 
device further comprising means for power conserva- 
tion contained within the timing control block; the device 
wherein the timing control block further comprises 
means for providing a signal which is delivered external 
to the device for synchronizing with a mechanical shut- 
ter; the device wherein the timing control block further 
comprises means for providing a signal which is deliv- 
ered external to the device for synchronizing with a 



strobe light; the device further comprising a Microcon- 
troller embedded within the sensor device, the Micro- 
controller further comprising interface means for user in- 
put; the device further comprising an ASIC designed for 

5 interfacing the Microcontroller to the remaining portions 
of the sensor device; the device further comprising an 
embedded memory associated with the ADC output; 
and the device of claim 1 9 wherein the embedded mem- 
ory further comprises a data buffering mechanism which 

10 allows different speeds of reads and writes to the mech- 
anism. 

[0035] The invention has been described In detail with 
particular reference to certain preferred embodiments 
thereof, but it will be understood that variations and 
is modifications can be effected within the spirit and scope 
of the invention. 

PARTS LIST 



a silicon substrate having a plurality of CMOS 
circuit formed thereon ; 

a pixel array having a plurality of rows and a 
plurality of columns formed within the sub- 
strate; 

a timing control logic block formed within the 
substrate; 
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An image sensor device comprising: 
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a row addressing circuit lormed within the sub- 
strate and operatively connected to each the 
pixel array and the timing control circuit, the row 
addressing circuit having a row bus that pro- 
vides address lines to each row in the pixel ar- 
ray; 

a column addressing circuit formed within the 
substrate; 

a pixel timing circuit formed within the sub- 
strate; 

a signal processing circuit contained within the 
substrate; and 

an interface circuit coupled to external compu- 
tational means for provision of commands used 
by the sensor to generate address and control 
signals to the sensor device, the interface cir- 
cuit being operatively coupled the timing con- 
trol logic, the pixel timing circuity the row ad- 
dressing circuit and the column addressing cir- 
cuit. 

2. The invention of claim 1 wherein the timing control 
logic block further comprises an electronic shutter- 
ing control circuit. 

3. The invention of claim 2 wherein the electronic shut- 
tering control circuit further comprises a circuit that 
is programmable responsive to an input From the 
interface circuit. 

4. The invention of claim 3 wherein the electronic shut- 
tering control circuit is responsive to electronic shut- 
tering mode data received from the interface circuit 
relating to one of the following: adjustment of elec- 
tronic shuttering mode; clearing pixels relative to 
frame time; integration time for each frame; or fluo- 
rescent light beating. 

5. The image sensor device of claim 1 further compris- 
ing a signal processing circuit coupled to the pixel 
array to receive data output from the array. 

6. The image sensor device of claim 5 wherein the sig- 
nal processing circuit further comprises a correlated 
double sampling circuit. 

7. The image sensor device of claim 5 wherein the sig- 
nal processing circuit Farther comprises a noise re- 
duction circuit. 

8. The image sensor device of claim 7 wherein the 
noise reduction circuit further comprises a fixed pat- 
tern noise removal circuit. 

9. The image sensor device of claim 5 wherein the sig- 
nal processing circuit is further comprises an analog 
to digital converter. 



10. An semiconductor image sensing device compris- 
ing: 

an imaging array containing a plurality of pixels 
5 formed on a semiconductor substrate; 

a timing control block formed on the substrate 
programmable via a serial communications in- 
terface formed on the substrate; 
means for generating signal required to ad- 
10 dress and readout the pixel values formed on 

the substrate; 

nneans form processing the pixel values toform 
digital words formed on the substrate; and 
means for delivering the digital words to exter- 
^5 nal devices. 
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